Skip to content

test: Add CI for npm-installed CLI#5542

Draft
backspace wants to merge 2 commits into
mainfrom
cli-parse-tests-cs-12083
Draft

test: Add CI for npm-installed CLI#5542
backspace wants to merge 2 commits into
mainfrom
cli-parse-tests-cs-12083

Conversation

@backspace

Copy link
Copy Markdown
Contributor

No description provided.

The boxel-cli suite only ever called command functions in-process inside
the monorepo, where pnpm's nested node_modules masks how the package
behaves once installed from npm. `boxel parse` shipped broken — glint
silently resolved nothing under npm's hoisted node_modules layout —
despite a fully green suite.

Add a subprocess-driven harness whose only per-context variable is the
binary under test (BOXEL_CLI_BIN):

- tests/helpers/run-boxel.ts drives the CLI as a subprocess (argv + env +
  stdin -> stdout/stderr/exit).
- tests/helpers/integration.ts seeds a JWT profile on disk so a spawned
  CLI authenticates via a temp HOME with no Matrix round-trip.
- scripts/run-cli-suite.ts installs the target and runs a test command
  against it: `pnpm pack` (resolves catalog:/workspace: specifiers) then
  `npm install` the tarball for real hoisting, or `npm install` from the
  registry for the published artifact.

Wire two contexts into CI: the PR gate runs the suite against the packed
npm install (test:cli:tarball), and the publish workflow smoke-tests the
just-published registry artifact (verify-unstable / verify-stable).

Add parse fixtures covering the reported resolution gaps plus a
deliberate type error that proves glint actually ran, and port
smoke.test.ts onto the harness.

This is the test/CI half of CS-12083. The parse fix is a follow-up, so
the parse fixtures are expected to fail against the installed binary
until it lands.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Realm Server Test Results

    1 files  ±0      1 suites  ±0   17m 29s ⏱️ +39s
1 880 tests ±0  1 880 ✅ ±0  0 💤 ±0  0 ❌ ±0 
1 959 runs  ±0  1 959 ✅ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit a77f4dc. ± Comparison against earlier commit 2a718d1.

Rewrite the tests/integration suite to drive the installed `boxel` binary
via runBoxel instead of calling command functions in-process, so the same
tests exercise the CLI as installed from npm (the tarball PR gate and the
published post-release smoke).

- Add createTestRealmViaCli(home): create a realm through `boxel realm
  create` and read its URL back from the profile the CLI wrote to disk.
- Each test seeds a profile on disk (createTestHome + setupTestProfile /
  setupJwtTestProfile), runs the command under test via runBoxel, and
  verifies realm/DB/checkpoint state in-process via reloadProfile(home).
- Text content rides stdin; binary content is staged to a temp --file.
- Commands with --json are asserted via res.json(); others via stdout
  (the CLI drops ANSI colors when stdout is not a TTY) and exit code.

A few tests stay in-process by necessity, coexisting with the subprocess
tests in the same files: white-box cases that mock/spy the internal fetch
(request-body shape, injected non-2xx responses) can't cross the process
boundary, and realm-watch is the long-running `watch start` daemon plus
the RealmWatcher internal API, which a subprocess can't drive without
weakening assertions.

Not run locally (needs the realm-server + Postgres + Matrix stack);
eslint clean across the suite. Expect CI iteration to settle argv and
output-assertion details.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant